Exploring a Continuous and Flexible Representation of the Lexicon

We aim at showing that lexical descriptions based on multifactorial and continuous models can be used by linguists and lexicographers (and not only by machines) so long as they are provided with a way to efficiently navigate data collections. We propose to demonstrate such a system. 1 Background and Motivations “You shall know a word by the company it keeps!” (Firth, 1957). This all too well-known citation motivates any lexicographic work today: it is widely accepted that word description cannot be achieved without the analysis of a large number of contexts extracted from real corpora. However, this is not enough. The recent success of deep learning approaches has shown that discrete representations of the lexicon are no longer appropriate. Continuous models offer a better representation of word meaning, because they encode intuitively valid and cognitively plausible principles: semantic similarity is relative, contextsensitive and depends on multiple-cue integration. At this point, one may say that it doesn’t matter if these models are too abstract and too complex for humans as they are used by machines. We think this argument is wrong. If continuous models offer a better representation of the lexicon, we must conceive new lexical databases that are usable by humans and have the same basis as these continuous models. There are arguments to support this view. For example, it has been demonstrated that semantic categories have fuzzy boundaries and thus the number of word meanings per lexical item is to a large extent arbitrary (Tuggy, 1993). Although this still fuels lots of discussions among linguists and lexicographers, we think that a description can be more or less fine-grained while maintaining accuracy and validity. Moreover, it has been demonstrated that lexical entries in traditional dictionaries overlap and different word meanings can be associated with a sole example (Erk and McCarthy, 2009), showing that meaning cannot be sliced into separate and exclusive word senses. The same problem also arises when it comes to differentiating between arguments and adjuncts. As said by Manning (2003): “There are some very clear arguments (normally, subjects and objects), and some very clear adjuncts (of time and ‘outer’ location), but also a lot of stuff in the middle”. A proper representation thus need to be based on some kind of continuity and should take into consideration not only the subject and the object, but also the prepositional phrases as well as the wider context. Some applications already address some of the needs of lexicographers in the era of big data, i.e. big corpora in this context. The most well-known application is the SketchEngine (Kilgarriff et al., 2014). This tool has already provided invaluable services to lexicographers and linguists. It gives access to a synthetic view of the different usages of words in context. For example, the SketchEngine can give a direct view of all the subjects or complements of a verb, ranked by frequency or sorted according to various parameters. By exploding the representation, this tool provides an interesting view of the lexicon. However, in our opinion, it falls short when it comes to showing the continuous nature of meaning. This work is licenced under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ Here we propose a system that combines the advantages of existing tools (a wide coverage database offering a synthetic view of a large vocabulary) with those of a dynamic representation. We focus on verbs since these lexical items offer the most complex syntactic and semantic behaviors. More specifically, we examine Japanese verbs, as Japanese is a language that presents a complex system of case markers that are generally semantically ambiguous. 2 Outline of our approach When building a verb lexicon, numerous challenges arise such as the notion of lexical item – that is, how many entries and subentries are necessary to describe the different meanings of a given verb? – and the distinction between arguments and adjuncts – that is, what complements are necessary to describe a particular meaning of a given verb? Following up on studies in natural language processing and linguistics, we embrace the hypothesis of a continuum between ambiguity and vagueness (Tuggy, 1993), and the hypothesis that there is no clear distinction between arguments and adjuncts (Manning, 2003). Although this approach has been applied and evaluated for Japanese, the theoretical framework to compute the argumenthood of a complement, or build the hierarchical structure of the lexical entries, is partially independent. We assume a list of verbal structures that have been automatically extracted from a large representative corpus. A verbal structure is an occurrence of a verb and its complements (expressed as syntactic dependencies); a complement is an ordered pair of a lexical head and a case marker. Computing the argumenthood of complements Following up on previous studies on the distinction between arguments and adjuncts (Manning, 2003; Merlo and Esteve Ferrer, 2006; Fabre and Bourigault, 2008; Abend and Rappoport, 2010), we propose a new measure of the degree of argumenthood of complements, derived from the famous TF-IDF weighting scheme used in information retrieval: argumenthood(v, c) = (1 + log count(v, c)) log |V | |{v′ ∈ V : ∃(v′, c)}| (1) where c is a complement (i.e. an ordered pair of a lexical head and a case particle); v is a verb; count(v, c) is the number of cooccurrences of the complement c with the verb v; |V | is the total number of unique verbs; |{v′ ∈ V : ∃(v′, c)}| is the number of unique verbs cooccurring with this complement. That is, we are dealing with complements instead of terms, and with verbs instead of documents. This measurement captures two important rules of thumb for distinguishing between arguments and adjuncts. The first part of the formula (1 + log count(v, c)) takes the idea that complements appearing frequently with a given verb tend to be arguments; the second part of the formula log |V | |{v′∈V :∃(v′,c)}| , that complements which appear with a large variety of verbs tend to be adjuncts. The proposed measure assigns a value between 0 and 1 to a complement – 0 corresponds to a prototypical adjunct; 1 corresponds to a prototypical argument – and thus model a continuum between arguments and adjuncts. Enriching verb description using shallow clustering A verbal structure corresponds to a specific sense of a given verb; that is, the sense of the verb is given by the complements selected by the verb. Yet a single verbal structure contains a very limited number of complements. So as to obtain a more complete description of the verb sense, we propose to merge verbal structures corresponding to the same meaning of a given verb into a minimal predicate-frame using reliable lexical clues. We call this technique shallow clustering. Our method relies on the principles that i) two verbal structures describing the same verb and having at least one common complement might correspond to the same verb sense, and that ii) some complements are more informative than others for a given sense. As for the second principle, the measure of argumenthood, introduced in the previous section, serves as a tool for identifying the complements which contribute the most to the verb meaning. Our method merges verbal structures in an iterative process – beginning with the most informative complements (i.e. complements yielding the highest argumenthood value) – as shown in Algorithm 1. Data: A set W of verbal structures (v,D), where v is a verb and D is a list of complements Result: A set W′ of minimal predicate-frames (v,D′) such that |W ′| 6 |W | W ′ ← ∅; foreach verb v in {v : ∃(v,D) ∈W} do /* Let C’ be the list of complements cooccurring with v sorted by argumenthood values in non-increasing order */ C ← {c : ∃(v,D) ∈W ∧ c ∈ D}; C ′ ← (c : c ∈ C ∧ argumenthood(v, C ′[i]) > argumenthood(v, C ′[i+ 1])); for i← 0 to length(C’)− 1 do /* Let D ′ be a subset of {D : ∃(v,D) ∈W} */ D′ ← ∅; foreach list of complements D in {D : ∃(v,D) ∈W} do if C ′[i] ∈ D then add D to D′; remove (v,D) from W ; end end foreach list of complements D in {D : ∃(v,D) ∈W} do if ∃X ∈ D′ such that D ⊂ X then add D to D′; remove (v,D) from W ; end end if |D′| > 2 then add the minimal predicate-frame (v,D′) to W ′; end end Algorithm 1: Shallow clustering of verbal structures. Modeling word senses through hierarchical clustering We propose to cluster the minimal predicateframes built during the shallow clustering procedure into a dendrogram structure. A dendrogram allows the definition of an arbitrary number of classes (using a threshold) and thus fits nicely with our goal of modeling a continuum between ambiguity and vagueness. A dendrogram is usually built using a hierarchical clustering algorithm, with a distance matrix as its input. So as to measure the distance between minimal predicate-frames, we propose to represent minimal predicate-frames as vectors which would then serve as arguments of a similarity function. Following previous studies on semantic composition, we suppose that “the meaning of a whole is a function of the meaning of the parts and of the way they are syntactically combined” (Partee, 1995) as well as all the information involved in the composition process (Mitchell, 2011). The following equation summarizes the proposed model of semantic composition: